第 3 章：調度 Pods 好夥伴 Labels
為什麼需要 Labels
當 Pod 數量越來越多時，管理的維度也會逐漸複雜，譬如如何將 Pod 在不同地區上部署，不同層級的 Pod 如何分開管理，如何決定哪個類型的 Pod，部署在相對應的 Node 上，都是我們在實際場景上需要考量的部分。 Labels 讓我們能對每個不同屬性的 Pod 都能貼上屬於他們自己的標籤、且將有不同種類的標籤 Pod 做分群管理
Labels 介紹
- 用來管理資源：Pods、Nodes
- 透過 Label Selectors 管理與選擇物件
- Labels 就是
一對具有辨識度的 key/value- "release" : "stable"，"release" : "qa"
- "enviroment": "dev"，"enviroment": "production"
- "tier": "backend", "tier": "frontend"
- "department": "enginnerting", "department": "marketing", "department": "finance"
若 Kubernetes Cluster 中的 Pod 物件帶有標籤，由上面幾個 labels 我們可以清楚知道，他們的功用是什麼、層級是什麼，他們服務的對象是哪些部門的人。Labels 有以下幾個特點
- 每個物件可以同時擁有許多個 labels (multiple labels)
- 可以透過 Selector，幫我們縮小要尋找的物件
- 目前 API 提供不再只是一個 key 對應一個 value (Equality-based requirement) 的關係，我們也可以使用 matchExpressions 來設定更有彈性的 Labels
Label Selector
- name = redis-slave
- env != production
- name in (redis-master,redis-slave)
- name not in (php-fronted)
動態新增 Labels
除了在定義檔一開始就加入 Labels 以外，我們也可以透過 kubectl label的指令，來為我們的 Pod 新增標籤，指令如下：
kubectl label pods my-pod env=production
pod "my-pod" labeled
kubectl get pods my-pod --show-labels
NAME READY STATUS RESTARTS AGE LABELS
my-pod 1/1 Running 0 1h app=webserver,env=production,tier=backend
若是再次查看 my-pod 的 Labels，就可以看到env=production被新增在 LABELS中。
實作：如何將 Pod 部署到特定的 Node 上
同樣我們也能將 Labels 的觀念套用在 Node 上。可以透過幫每個 Node 貼標籤讓 Pod 運行在特定的 Node 上。以下我們將做兩件事情：
- 在 Node 上貼上標籤
- 在 Pod 新增
nodeSelector定義
在 Node 上貼上標籤
在新增 label 前，先檢查一下目前 Node 有的 labels，
kubectl get nodes —show-labels
my-pod2.yaml
apiVersion: v1
kind: Pod
metadata:
name: my-pod2
labels:
app: webserver
tier: backend
annotations:
version: latest
release_date: 2017/12/28
contact: zxcvbnius@gmail.com
spec:
containers:
- name: pod-demo
image: zxcvbnius/docker-demo
ports:
- containerPort: 3000
nodeSelector:
hardware: high-memory
創建一個新的 my-pod2 物件
$ kubectl create -f my-pod2.yaml
pod "my-pod2" created
kubectl get pods
NAME READY STATUS RESTARTS AGE
my-pod 1/1 Running 0 13m
my-pod2 0/1 Pending 0 3m17s
kubectl describe pod my-pod2
Name: my-pod2
Namespace: default
Priority: 0
Service Account: default
Node: <none>
Labels: app=webserver
tier=backend
Annotations: contact: [zxcvbnius@gmail.com](mailto:zxcvbnius@gmail.com)
release_date: 2017/12/28
version: latest
Status: Pending
IP:
IPs: <none>
Containers:
pod-demo:
Image: zxcvbnius/docker-demo
Port: 3000/TCP
Host Port: 0/TCP
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-j2n5p (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
kube-api-access-j2n5p:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: hardware=high-memory
Tolerations: [node.kubernetes.io/not-ready:NoExecute](http://node.kubernetes.io/not-ready:NoExecute) op=Exists for 300s
[node.kubernetes.io/unreachable:NoExecute](http://node.kubernetes.io/unreachable:NoExecute) op=Exists for 300s
Events:
Type Reason Age From Message
---
Warning FailedScheduling 4m9s default-scheduler 0/3 nodes are available: 1 node(s) had untolerated taint {[node-role.kubernetes.io/control-plane:](http://node-role.kubernetes.io/control-plane:) }, 2 node(s) didn't match Pod's node affinity/selector. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling..
- 新增一個
label到目前的nodek8s-wrk-1上
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-msr-1 Ready control-plane 38m v1.28.0
k8s-wrk-1 Ready <none> 37m v1.28.0
k8s-wrk-2 Ready <none> 37m v1.28.0
kubectl label node k8s-wrk-1 hardware=high-memory
node/k8s-wrk-1 labeled
kubectl get node --show-labels
NAME STATUS ROLES AGE VERSION LABELS
k8s-msr-1 Ready control-plane 40m v1.28.0 [beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-msr-1,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node.kubernetes.io/exclude-from-external-load-balancers=](http://beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-msr-1,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node.kubernetes.io/exclude-from-external-load-balancers=)
k8s-wrk-1 Ready <none> 38m v1.28.0 [beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,hardware=high-memory,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-wrk-1,kubernetes.io/os=linux](http://beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,hardware=high-memory,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-wrk-1,kubernetes.io/os=linux)
k8s-wrk-2 Ready <none> 38m v1.28.0 [beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-wrk-2,kubernetes.io/os=linux](http://beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-wrk-2,kubernetes.io/os=linux)
- 可以發現 k8s-wrk-1 上多了一個 label，這時若是我們再去查看 my-pod 的狀態會發現
kubectl get pods
NAME READY STATUS RESTARTS AGE
my-pod 1/1 Running 0 16m
my-pod2 1/1 Running 0 6m52s
kubectl describe pod my-pod2
Events:
Type Reason Age From Message
Warning FailedScheduling 2m17s (x2 over 7m24s) default-scheduler 0/3 nodes are available: 1 node(s) had untolerated taint {[node-role.kubernetes.io/control-plane:](http://node-role.kubernetes.io/control-plane:) }, 2 node(s) didn't match Pod's node affinity/selector. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling..
Normal Scheduled 84s default-scheduler Successfully assigned default/my-pod2 to k8s-wrk-1
Normal Pulling 84s kubelet Pulling image "zxcvbnius/docker-demo"
Normal Pulled 66s kubelet Successfully pulled image "zxcvbnius/docker-demo" in 18.032s (18.032s including waiting)
Normal Created 66s kubelet Created container pod-demo
Normal Started 66s kubelet Started container pod-demo
Annotations 介紹
如果是沒有識別用途的標籤，Kubernetes 也提供了我們一個 Annotations 元件。以 Pod 為例，我們可以在 Pod 的 Annotations 紀錄該 Pod 的發行時間，發行版本，聯絡人email。Annotations 主要是方便開發者、以及系統管理者管理上的方便，不會直接被 Kubernetes 使用
my-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: my-pod
labels:
app: webserver
tier: backend
annotations:
version: latest
release_date: 2017/12/28
contact: zxcvbnius@gmail.com
spec:
containers:
- name: pod-demo
image: zxcvbnius/docker-demo
ports:
- containerPort: 3000
- 透過
kubectl create來創建 一個新 Pod 物件
kubectl create -f ./my-pod.yaml
pod "my-pod" created
接著我們可以透過kubectl get pods 來取得目前 Pod 的資訊，若是在指令後面加上--show-labels則也會顯示 Pod的labels
kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
my-pod 1/1 Running 0 81s app=webserver,tier=backend
可以看到我們剛剛指定的labels。我們也可以透過 kubectl describe 去查看該 Pod 的 annotations 有哪些
kubectl describe pods my-pod
Name: my-pod
Namespace: default
Priority: 0
Service Account: default
Node: k8s-wrk-2/10.0.1.203
Start Time: Wed, 10 Jan 2024 08:18:41 +0000
Labels: app=webserver
tier=backend
Annotations: contact: zxcvbnius@gmail.com
release_date: 2017/12/28
version: latest
Status: Running
IP: 172.16.2.2
IPs:
IP: 172.16.2.2
Containers:
pod-demo:
Container ID: containerd://3edde75ceac993a97fc67f36a220b9676d9919910d76eb65d8881e8dc56b0db0
Image: zxcvbnius/docker-demo
Image ID: docker.io/zxcvbnius/docker-demo@sha256:a3c6b9ab7356438f873f90c2af170a41cb32cadc01ab3d11f72101daa3abdf18
Port: 3000/TCP
Host Port: 0/TCP
State: Running
Started: Wed, 10 Jan 2024 08:19:00 +0000
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mqfqq (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-mqfqq:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 108s default-scheduler Successfully assigned default/my-pod to k8s-wrk-2
Normal Pulling 107s kubelet Pulling image "zxcvbnius/docker-demo"
Normal Pulled 89s kubelet Successfully pulled image "zxcvbnius/docker-demo" in 18.229s (18.229s including waiting)
Normal Created 89s kubelet Created container pod-demo
Normal Started 89s kubelet Started container pod-demo